In recent years, it is usual that ordinary users collect and accumulate a great deal of contents. This is in part because there have been an increasing number of opportunities for users to take photos or video, with the prevalence of digital cameras, digital video cameras and camera-equipped mobile phones. In addition, contents are collected and accumulated by recording various video content data, such as television broadcast program data, with use of DVD recorders, hard disk recorders or personal computers, and by storing data, such as still images, moving pictures and audio, which are provided on Web sites of the Internet, in storage devices such as hard disks.
Various search methods have been proposed as methods for searching accumulated contents for necessary data. Jpn. Pat. Appln. KOKAI Publication No. 2000-276484 discloses an image search apparatus which can search for an image in an image database. This image search apparatus can search for images by taking into account the time, as well as image feature amounts, by grouping images which have been generated or modified at mutually close time points. In addition, Jpn. Pat. Appln. KOKAI Publication No. 2006-48711 discloses an image search system which generates an image, in which extracted images are laid out, by taking the degree of attention into account. This image search system can search for images by using, as search keys, the size of image, the grouping attribute and the degree of attention, and can generate an image in which the searched images are laid out based on the degree of attention.
In the image search apparatus of KOKAI Publication No. 2000-276484, images are searched for by using characteristics which can objectively be detected from each image, for instance, the date/time of capturing images, image features, and persons and objects appearing on images. Similarly, in the image search system of KOKAI Publication No. 2006-48711, images are searched for by using of objective characteristics of images, such as grouping attributes based on the size of image, the image features and a person or object appearing on images, and the objective degree of attention of images.
However, if contents (images, etc.), which are the objects of search, are collected by a specific user, it is highly possible that the contents are a set of contents which are collected by some intent based on the subjective view of the user. Thus, if such contents, which are collected by a specific user, to be searched for, it is possible that contents, which are suited to the subjective view or intent of the user, cannot be searched for by the search based on the objective characteristic.
For example, if the user searches for contents by using an arbitrary person as a search key, it is not difficult to retrieve a photo in which the person is clearly captured, by executing a face image recognition process on the contents. However, it is difficult to search for contents relating to the person, for example, a photo in which the person appears in a small size, a photo of a scene relating to the person, a photo of another person relating to this person, or a photo of an object relating to this person. Although there arises no problem if only face photos of the person who is used as the search key are to be retrieved, it is possible, for example, that a monotonous album or slide show, in which only face photos are displayed, may be generated using photos extracted by such retrieval.